The present invention generally relates to interactive hypermedia systems and more particularly to a method and system for creating hyperlinks from selected items (e.g., words, pictures, foot notes, symbols, icons) on hard-copy documents to locally or remotely accessible servers, for highlighting by means of a light emitting source the position of each selected item, and for triggering anyone of said hyperlinks simply by touching the hard-copy document over said highlighted items.
Internet
The Internet is a global network of computers and computers networks (the xe2x80x9cNetxe2x80x9d). The Internet connects computers that use a variety of different operating systems or languages, including UNIX(trademark), DOS(trademark), Windows(trademark), Macintosh(trademark), and others. To facilitate and allow the communication among these various systems and languages, the Internet uses a language referred to as TCP/IP (xe2x80x9cTransmission Control Protocol/Internet Protocolxe2x80x9d). TCP/IP protocol supports three basic applications on the Internet:
transmitting and receiving electronic mail,
logging into remote computers (the xe2x80x9cTelnetxe2x80x9d), and
transferring files and programs from one computer to another (xe2x80x9cFTPxe2x80x9d or xe2x80x9cFile Transfer Protocolxe2x80x9d).
World Wide Web
With the increasing size and complexity of the Internet, tools have been developed to help find information on the network, often called navigators or navigation systems. Navigation systems that have been developed include standards such as Archie(trademark), Gopher(trademark) and WAIS(trademark). The World Wide Web (xe2x80x9cWWWxe2x80x9d or xe2x80x9cthe Webxe2x80x9d) is a recent superior navigation system. The Web is:
an Internet-based navigation system,
an information distribution and management system for the Internet, and
a dynamic format for communicating on the Web.
The Web seamlessly, for the use, integrates format of information, including still images, text, audio and video. A user on the Web using a graphical user interface (xe2x80x9cGUIxe2x80x9d, pronounced xe2x80x9cgooeyxe2x80x9d) may transparently communicate with different host computers on the system, different system applications (including FTP and Telnet), and different information formats for files and documents including, for example, text, sound and graphics.
Hypermedia
The Web uses hypertext and hypermedia. Hypertext is a subset of hypermedia and refers to computer-based xe2x80x9cdocumentsxe2x80x9d in which readers move from one place to another in a document, or to another document, in a non-linear manner. To do this, the Web uses a client-server architecture. The Web servers enable the user to access hypertext and hypermedia information through the Web and the user""s computer. (The user""s computer is referred to as a client computer of the Web Server computers.) The clients send requests to the Web Servers, which react, search and respond. The Web allows client application software to request and receive hypermedia documents (including formatted text, audio, video and graphics) with hypertext link capabilities to other hypermedia documents, from a Web file server.
The Web, then, can be viewed as a collection of document files residing on Web host computers that are interconnected by hyperlinks using networking protocols, forming a virtual xe2x80x9cwebxe2x80x9d that spans the Internet.
Uniform Resource Locators
A resource of the Internet is unambiguously identified by a Uniform Resource Locator (URL), which is a pointer to a particular resource at a particular location. A URL specifies the protocol used to access a server (e.g. HTTP, FTP, . . . ), the name of the server, and the location of a file on that server.
Hyper Text Transfer Protocol
Each Web page that appears on client monitors of the Web may appear as a complex document that integrates, for example, text, images, sounds and animation. Each such page may also contain hyperlinks to other Web documents so that a user at a client computer using a mouse may click on icons and may activate hyperlink jumps to a new page (which is a graphical representation of another document file) on the same or a different Web server.
A Web server is a software program on a Web host computer that answers requests from Web clients, typically over the Internet. All Web servers use a language or protocol to communicate with Web clients which is called Hyper Text Transfer Protocol (xe2x80x9cHTTPxe2x80x9d). All types of data can be exchanged among Web servers and clients using this protocol, including Hyper Text Markup Language (xe2x80x9cHTMLxe2x80x9d), graphics, sound and video. HTML describes the layout, contents and hyperlinks of the documents and pages. Web clients when browsing:
convert user specified commands into HTTP GET requests,
connect to the appropriate Web server to get information, and
wait for a response. The response from the server can be the requested document or an error message.
After the document or an error message is returned, the connection between the Web client and the Web server is closed.
First version of HTTP is a stateless protocol. That is, with HTTP there is no continuous connection between each client and each server. The Web client using HTTP receives a response as HTML data or other data. This description applies to version 1.0 of HTTP protocol, while the new version 1.1 breaks this barrier of stateless protocol by keeping the connection between the server and client alive under certain conditions.
Browser
After receipt, the Web client formats and presents the data or activates an ancillary application such a sound player to present the data. To do this, the server or the client determines the various types of data received. The Web Client is also referred to as the Web Browser, since it in fact browses documents retrieved from the Web Server.
Interactive Access To Multimedia Services
Interactive electronic services, video-on-demand, and the World Wide Web are providing access to an increasing offering of movies, shopping information, games, multimedia documents, electronic commerce and many other services. A major problem in using these systems is to browse the enormous variety and quantity of possible choices to discover what is available, and to make a selection. By example, when surfing on the Web, a conventional method to navigate across many pages of hypertext documents consists of using search tools or invoking bookmarked links to the different required topics. When surfing on video-on-demand services, a conventional method to navigate is to surf on channels. Advertisements on preview channels are used as entry points to other movies. Users can navigate and make selections from a remote control using hierarchical menus. Obviously, these approaches does not allows a rapid access and browse of the thousands of multimedia documents that are available on the Web or interactive TV.
Present invention is based on the recognition of two significant facts:
People are very skilled at browsing through paper catalogs, magazines, newspapers, maps and books by flipping through the pages and glancing at pictures and text.
A collection of printed color photographs can be much easily and quickly browsed than a sequence of computer screens. Paper has a number of useful properties:
paper is easy to read, mark, and manipulate;
paper is portable, familiar and can be easily distributed.
Many electronic systems attempt to replace paper by providing many advantages such as, for example, a better access to multimedia services. But, however most users prefer to work with paper. It is difficult to foresee, for example, the replacement in the future, of paper catalogs by electronic catalogs (e.g., by Web accessible catalogs). Publication entitled xe2x80x9cThe Last Bookxe2x80x9d, IBM Systems Journal, Vol 36, No. 3 Vol 36, No. 3-1997, by J. Jacobson, B. Comiskey, C. Turner, J. Albert, and P. Tsao of the MIT Media Laboratory, compares printed books and computer screens in the following terms:
xe2x80x9cA book represents a fundamentally different entity than a computer screen in that it is a physical embodiment of a large number of simultaneous high-resolution displays. When we turn the page, we do not lose the previous page. Through evolution the brain has developed a highly sophisticated spatial map. Persons familiar with a manual or textbook can find information that they are seeking with high specificity, as evidenced by their ability to remember whether something that was seen only briefly was on the right side or left side of a page, for instance. Furthermore their haptic connection with the brain""s spatial map comprises a highly natural and effective interface, when such information is embodied on actual multiple physical pages.
Another aspect of embodying information on multiple, simultaneous pages is that of serendipity and comparison. We may leaf through a large volume of text and graphics, inserting a finger bookmark into those areas of greatest interest. Similarly, we may assemble a large body of similar matter in order to view elements in contrast to one another, such as might be done to determine which of a particular set of graphical designs is most satisfyingxe2x80x9d.
Out of those advantages the most important problem, of course, with traditional printed books is that they cannot be changed, amended, updated nor completed.
During the last years, due mainly to the widespread use of personal computers and the universal access of millions of users to the World Wide Web, the xe2x80x9cmultimedia publishingxe2x80x9d has veritably exploded. Due to the widespread penetration of CD-ROM drives an enormous amount of multimedia titles combining text, images and sounds, are now accessible to owners of personal computers. In this evolution, an incredible amount of hypermedia information is today accessible via the Internet on the World Wide Web.
Even when the public""s enthusiasm for new computer-based multimedia services has been seen by many analysts as a threat to the conventional forms of hard-copied publishing, particularly book publishing, the real fact is that reading a book cannot be compared with reading an electronic media. Reading paper remains preferable for most people, whether they are familiar with computers or not.
Touching directly the objects we have around is one of the simplest, most instinctive, and universal human actions.
Finger pointing and touching are the most natural form of human/machine interface. The action of touching is so simple and natural that navigating by means of touch screens require no training and no learning. In fact the pervasiveness robustness and versatility of the xe2x80x9ctouch technologyxe2x80x9d is transforming the way people are living, working, learning, and playing. The xe2x80x9ctouch technologyxe2x80x9d is successfully used in many different applications, by example:
In industrial environments: Environmentally-robust touch screens are increasing productivity under hazardous and hostile conditions that would cripple a standard Personal Computer and keyboard.
In hospitals: Touch input helps doctors to prescribe medications to patients faster by allowing handwritten prescriptions.
In retail locations: Interactive, through-the-window displays let customers shop whenever they want, even when a store is closed.
In mobile and consumer devices: Touch and stylus input is the widely accepted input method for portable, and other mobile devices.
At tourist destinations: User-friendly kiosks are a cost-effective way to help travelers to get information and make their own reservations.
The two main advantages of touching are:
Simplicity: Touching with the fingertip is the simplest and more intuitive form of pointing and selecting an item, and
Versatility: Touching is particularly adapted to applications where the use of a keyboard, a mouse or an optical pencil or stylus is not practical or is not well adapted to the user""s service or comfort.
Traditionally, a touch panel is integrated into the computer display. The touch panel and the display forms a combination called xe2x80x9ctouch screenxe2x80x9d. The input device is integrated into the monitor, so no space is wasted, and the interaction of the user with the system is made easier. The system guides the user by showing different choices in the form of icons displayed on the screen. When the user touches the icon of its choice, the associated action is executed.
Therefore, there is a real need to provide the user with new systems and methods for improving printed texts with electronically stored data in the form of images, sounds and/or additional text.
U.S. Pat. No. 5,957,697 entitled xe2x80x9cPrinted Book Augmented with an Electronic Virtual Book and Associated Electronic Dataxe2x80x9d, discloses a system and method for augmenting a printed text with electronically stored data in the form of images, sounds and/or additional text where a printed book comprising a plurality of pages of text is emulated by an electronic virtual book, that mimics the appearance of the printed book. This invention is based on the duplication of the paper book, on the creation of an electronic book over which links to hypermedia are defined and can be selected by the user.
U.S. Pat. No. 5,903,729 entitled xe2x80x9cMethod, System, and Article of Manufacture for Navigating to a Resource in an Electronic Networkxe2x80x9d, discloses a method for navigating on an electronic network. The method comprises the steps of
forming an electronic image of an object having a plurality of markable regions associated with a plurality of electronic resources;
processing the electronic image to detect which of the markable regions associated with the plurality of electronic resources is marked;
generating a list comprising at least one link to at least one of the electronic resources whose associated markable region is marked, and
displaying a display screen based upon the list.
This invention requires image scanning and image processing means to read marked regions on hard-copied documents.
A similar approach using optical image scanners to access multimedia services is described in U.S. Pat. No. 5,640,193 entitled xe2x80x9cMultimedia Service Access by Reading Marks on an Objectxe2x80x9d. This patent discloses an apparatus and a method to enable a user to control the selection of electronic multimedia services by means of a scanner for reading marks on an object and for communicating a request signal, having an object code representing the read marks, to a user interface.
U.S. Pat. No. 5,495,581 entitled xe2x80x9cMethod and Apparatus for Linking a Document with Associated Reference Information Using Pattern Matchingxe2x80x9d also uses image scanning and pattern matching techniques. This patent discloses an apparatus for linking a portion of a document with associated reference information, wherein the linked portion is designated by a predetermined attribute of the received document image, using among several other, a device for electronically scanning the electronic representation of the document image to locate said predetermined attribute of the document""s image.
U.S. Pat. No. 5,905,251 entitled xe2x80x9cHand-held Portable WWW Access Terminal with Visual Display Panel and GUI-based WWW Browser Program Integrated with Bar Code Symbol Reader in a Hand-supportable Housingxe2x80x9d discloses a portable hand-held WWW access terminal for accessing HTML-encoded documents located on the WWW. The terminal includes a bar code symbol reader in a hand-supportable housing for reading URL-encoded symbols specifying the location of HTML-encoded documents stored in information servers connected to the Internet and supporting the TCP/IP standard. This invention requires the marking of physical documents with bar code symbols and requires bar code readers to trigger hyperlinks.
Finally, a different approach for having an access to mutimedia services from physical documents is proposed in U.S. Pat. No. 5,624,265 entitled xe2x80x9cPrinted Publication Remote Control for Accessing Interactive Mediaxe2x80x9d. This patent discloses a remote control system for an interactive media comprising a printed publication (such as a book, a magazine or a catalog), and one or a plurality of buttons physically attached to the printed publication to allow users to remotely control use of associated electronic content by a host device.
None of the methods referenced in the prior art describes a system that would enable a user to get access to hypermedia services simply by touching with a fingertip an original, unmarked, hard-copied document (e.g., over an original edition of a rare ancient book).
From the prior art analysis, there is a real, uncovered need, for a system and a method for selecting and accessing multimedia information and/or services from different items (i.e., words, icons, figures, foot notes, etc.) printed on a book, a hard-copy publication or, more generally, on any type of physical document:
simply by touching the document with the fingertip, while, at the same time,
maintaining the integrity of the document.
There is a need for not modifying a document using special marks or fonts to locate, identify or visualize the different items the user can select on the document. It would desirable to highlight these items while preserving the integrity of the physical document.
Aspects of the Invention
It is an aspect of the present invention to improve the current systems and methods of selecting and accessing electronic multimedia information or services.
It is another aspect of the present invention to select and to access electronic multimedia information or services directly from a physical document.
It is another aspect of the present invention to select and access electronic multimedia information or services simply by touching items on a physical document.
It is another aspect of the present invention to create hyperlinks between items on the physical document and multimedia information or services located on servers.
It is another aspect of the present invention to highlight said items while preserving the integrity of the document.
The present invention discloses a method and system of selecting and accessing information or services by touching hyperlinked items on a physical document. The method, for use in a user system, comprises the steps of:
identifying a physical document, said physical document comprising one or a plurality of pages;
identifying a page of said physical document, said page comprising one or a plurality of predefined hyperlinked items;
identifying position of hyperlinked items comprised in said identified page referring to a hyperlink table associated with said identified document, said hyperlink table comprising for each page of the document, a list of hyperlink items, and for each hyperlink item, an indication of its position on said page;
sending for visualization on an opto-touch foil, the position of said hyperlinked items, said opto-touch foil being connected to the user system.
The method comprises the further steps of:
determining the position of a point pressed on said opto-touch foil; said opto-touch foil being placed and aligned over or under the identified page of the physical document; each one of hyperlinked items on said page corresponding to a visualized position on said opto-touch foil; said opto-touch foil being pressed at a point corresponding to a selected hyperlinked item;
identifying the selected hyperlinked item corresponding to the position of the point pressed on said opto-touch foil referring to the hyperlink table, said hyperlink table comprising an indication of the position of each hyperlinked item on the identified page;
identifying and locating information or service associated with the selected hyperlinked item referring to the hyperlink table, said hyperlink table comprising for each hyperlinked item of each page of the document the identification and location of the information or service associated with the hyperlinked item;
acessing the information or service associated with the selected hyperlinked item.
The present invention discloses a method and system of creating hyperlinks, by touching hyperlinked items on a physical document, for use in a user system. The method comprises the steps of:
creating a hyperlink table for a physical document; said physical document comprising one or a plurality of pages;
receiving and storing in said hyperlink table an identification of the physical document;
for each page of said physical document:
receiving and storing in said hyperlink table an identification of the page and an identification of hyperlinked items defined by the user on said page;
receiving and storing in said hyperlink table identification and location of information or service associated with each defined hyperlinked item;
determining the position of points pressed on a opto-touch foil; said opto-touch foil being placed and aligned over or under the page of the physical document; said opto-touch foil being pressed at points corresponding to the position of said defined hyperlinked items;
storing the position of the points pressed in the hyperlink table, said hyperlink table comprising for each hyperlinked item, an indication of its position on the page.
The foregoing, together with other aspects, features, and advantages of this invention can be better appreciated with reference to the following specification, claims and drawings.